
\chapter{Syntax and semantics}

\label{chapter:semantics}

This chapter gives an overview of {\bf T} as a programming language.

The {\bf T} language logically comprises three distinct components:
\begin{itemize}
\item[{}]an \iix{external representation} for objects as character sequences,

\item[{}]a \iix{core language} - syntax and semantics for expressions, and

\item[{}]a \iix{standard environment} - the behavior of the objects
which are values of system variables.
\end{itemize}

\section{External representation}


Like other Lisp dialects, and in contrast with most other
programming languages, {\bf T} has two different kinds of syntax: the
\iix{external representation} by which objects (data) are represented as a
linear sequence of characters, and an \iix{expression syntax} by which
these objects can be understood as programs.  \index{objects}
That is, the meaning of a {\bf T} program represented as characters in a file
or other external storage medium must be determined in two stages:
first, by mapping the characters to objects, and then by interpreting
these objects as executable programs.

The external syntax of {\bf T} is very similar to that of other Lisp
dialects, and is discussed in detail at appropriate places in this manual.
The following gives only the most cursory description.
Characters may be classified according to their lexical properties; the
two most important distinctions are between {\it constituent\/} and
{\it delimiter\/} characters, and between {\it read-macro\/} characters and
non-read-macro characters.  Alphabetic, numeric, and some special
characters (e.g. {\tt -} and {\tt \$}) are constituent characters;
whitespace and some special characters (e.g. left and right parenthesis) are
delimiter characters.  Some special characters such as {\tt (} and '
are read-macro characters; these introduce special syntactic constructs
which are idiosyncratic to the particular character introducing them.

A delimited sequence of consecutive constituent characters represents
either a \iix{number} or a \iix{symbol}.  For example, {\tt 255} represents
the integer 255, and {\tt APPEND} and {\tt append} both represent the
symbol {\tt APPEND}.  Here, and throughout the manual, the term
{\it symbol\/} is being used in a technical manner to refer a particular
kind of named object (see section \ref{section:symbols}).

Balanced parentheses with a sequence of (representations of) objects
between them represent a list of the (represented) objects.  For
example, the characters {\mbox {\tt (A B (C 12) D)}} represent a list of four
objects, three of which are symbols, and one of which (the third one) is
a list of two objects, the symbol {\tt C} and the integer 12.
Parentheses which enclose no objects represent an empty list.

There are also external representations for strings and characters, as
well as for some kinds of objects which are syntactically illegal
as expressions (for example, vectors).  Not all objects have external
representations, however.

\section{Core language}

\label{section:coreLanguage}
The core language is described in terms of a hypothetical machine
(called an \iix{evaluator}) which executes {\bf T} code directly.  In
practice, the existing {\bf T} implementations have (at least) two
evaluators, both of which perform evaluation as a two-stage process
consisting of compilation (syntactic and semantic analysis) followed by
interpretation.

{\it Evaluation\/}\index{evaluation} is a process whereby an object called an {\it expression\/}
\index{expressions} (the term {\it form\/}\index{forms} is used synonymously
with {\it expression\/}) is mapped to another object, called its
{\it value\/}.  Evaluation occurs in the context of a particular
{\it variable environment\/}\index{environments}; see below.  The expression
is said to \iix{yield} its value.
\index{variables}

The evaluation mapping is not a purely mathematical mapping, since in
some cases the evaluation of an expression may depend not only on the
variable environment but on the \iix{state} of the running {\bf T}
system, or on the state of the world outside it; and evaluation may
cause changes in the state of system or the world, which may, in turn,
affect future evaluations.  These state changes are called
\iix{side-effects}.

The rules by which an object is evaluated are as follows:

{\it Self-evaluating literals:\/}\index{literals}
All numbers, strings, and characters are syntactically valid
expressions which, when evaluated, yield themselves.
For example,
\begin{codexenv}
    \begin{tabular}{lll}
-2102        & $\Longrightarrow$\   & -2102 \\
{\#$\backslash$}M         &  $\Longrightarrow$\   & {\#$\backslash$}M \\
"A string."  & $\Longrightarrow$\  &  "A string."
\end{tabular}
    \end{codexenv}

The notation ``{\it expression\/} $\Longrightarrow$\  {\it value\/}'' means that
{\it expression\/}, when evaluated, yields {\it value\/}.  (Note also that we are
making use of the external object syntax itself as a notational device:
``the object {\tt -2102}'' could be said more precisely as ``an object
externally represented by the characters ``{\tt -2102}''.''  Just as a
program should never be confused with an object which represents it, an
object should never be confused with a sequence of characters
which notates it.)

{\it Symbols:\/}\index{symbol}
As evaluable expressions, symbols are interpreted as variable
references.  A
symbol evaluates to its value according to the current variable
environment.  For example, in an environment in which the variable
{\tt DELTA} has the value {\tt 15}, evaluating the expression
{\tt DELTA} will yield {\tt 15}.
\index{variables}

Symbols have many uses other than as names for variables.
It is important not to confuse the use of a symbol as a datum
manipulated by a program with the occurrence of a symbol
as a variable reference in a program.  Symbols do not have values
{\it a priori\/}; variables only have values by virtue of the context
in which they occur.

{\it Lists:\/}\index{lists}
Non-empty lists are classified either as \iix{calls} or as \iix{special
forms}.  If the first element of the list is a symbol, and the symbol is
a \iix{reserved word}, then the list is a special form; otherwise it is
considered to be a {\it call\/}.  {\bf T} reserved words
, for example, the symbols {\tt QUOTE}, {\tt IF},
and {\tt LAMBDA}.  (However, see section \ref{section:syntaxTables}.)

{\it Special forms:\/}\index{special forms}
The syntax and semantics of a special form are
idiosyncratic to the reserved word which introduces it; descriptions of
the meaning of expressions introduced by the various reserved words are
therefore distributed throughout the manual.

{\it Calls:\/}\index{calls}
\label{CallSemantics}
Calls are evaluated as follows: the elements of the list (including the
first) are evaluated, {\bf in an undefined order}.  The first must evaluate
to a procedure; this procedure is {\it applied\/} \index{application} to the
rest of the values, which are called \iix{arguments}.
(The verbs {\it call\/} and {\it invoke\/} mean the same as {\it apply\/}.)
\index{calling} \index{invoking}

The evaluation order of arguments in a combination is undefined.
{\sc orbit} produces
code in which the evaluation order of arguments is undefined
and not necessarily left to right.

Particularly insidious bugs have resulted from {\tt LET},
see page \pageref{LET},
forms whose clauses contain order dependent side effects.  
{\tt LET$^*$\ } should be used to ensure
sequential evaluation order.

\section{The standard environment}

\label{section:standardEnvironment}
The standard environment corresponds to what is usually known as the
``run-time library'' in other language environments such as C or Pascal.

New variable environments may be introduced in various ways (see chapter
\ref{chapter:environments}), but a {\bf T} system is obliged to supply one
standard environment in which system variables are bound to system
procedures and constants, as defined by this manual.  Program execution
typically occurs in an environment inferior to this standard environment
(see section \ref{STANDARD-ENV}), so that these objects are easily
accessible as values of lexically apparent variables.  For example,
in this standard environment, the variable {\tt CONS} has a certain
procedure as its value.

For the most part, the values of system variables are procedures, and
therefore the only behavior of interest is what they do when called.
(See the discussion of calls, above.)  In other cases, values are objects
such as numbers or symbols.

A representation of the standard environment is available as the value
of {\tt STANDARD-ENV} (page \pageref{STANDARD-ENV}).

\section{Undefined}

\label{section:undefinedSemantics}

The term \iix{undefined} is used in two different ways in this manual.
An expression may {\it yield an undefined value\/}, in which case it yields
some value, the particular value not being defined by this manual.  In
such cases it is unwise to depend on this value having any particular
characteristics, for example, it being null, or not a number, or
whatever.  The evaluation of the expression and the creation of the
undefined value are not in error; it is the use to which this value is
put that may lead to problems.  Expressions yielding undefined values
are generally useful only for any side-effects they cause.

On the other hand, the evaluation of an expression may {\it have an
undefined effect\/}, in which case an implementation will endeavor to
signal an error condition, permitting a user to take appropriate action.
For example, a call to a non-procedure, or adding two symbols together,
have undefined effects.  An implementation is not {\it obliged\/} however,
to signal an error, and in fact it may be in the interest of efficiency
to avoid the overhead of detecting circumstances under which undefined
effects will happen.

The two procedures {\tt UNDEFINED-VALUE} and {\tt UNDEFINED-EFFECT}
are used for expository purposes in examples throughout this manual;
they are described in section \ref{section:undefined}.

\section{Multiple values}

\label{section:multipleValues}

{\bf T} now supports multiple return values.  This makes
procedure call and return uniform, in the sense that a procedure
can be invoked with zero or more values and can return zero
or more values. See page \pageref{subsection:multipleReturnValues}.

Like procedures, continuations have certain expectations about the
number of arguments (values) that will be delivered to them.  It is an
error if more or fewer values are delivered than expected.

The idiom {\mbox {\tt (RETURN)}} is useful for procedures that return
an undefined value and many of the system procedures whose value(s)
is undefined now return no value.  However,  the procedure
{\mbox {\tt undefined-value}} may provide a more informative error
message.

Other forms,  such as {\mbox {\tt CATCH}} and {\mbox {\tt RET}},
have been extended to allow multiple return
values.
